Why the Normal Distribution?

This short note explains in simple terms why the normal distribution is so ubiquitous in pattern recognition applications. As we will see, the answer comes from information theory: when we compress a data set by keeping only the mean and variance of the problem’s classes, the distribution which allows us to keep on working probabilistically, while making minimal assumptions about the data, is, precisely, the normal distribution. Why the normal distribution? The geek store offers a t-shirt with the legend “let < 0 . . .”. That is the complete punch line. Similarly, in pattern recognition applications people would raise their heads if someone said “let us modell the data clusters, about which we know almost nothing, by using an xy distribution”, where “xy” is not the word “normal”. In other words: you better have good reasons to propose modelling data clusters with anything else than a Gaussian, or at least a mixture of Gaussians. But then, why is the normal distribution so prevalent in pattern recognition applications? The classical book by Duda & Hart, for example, starts right with the Bayes rule and the multivariate normal distribution [1]. The Gaussian (normal) distribution, is used in many pattern recognition problems as an easy way of modelling the probability density of experimental data.